{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 题目：  \n",
    "这份数据集是金融数据（非原始数据，已经处理过了），我们要做的是预测贷款用户是否会逾期。表格中 \"status\" 是结果标签：0表示未逾期，1表示逾期。\n",
    "### 要求：  \n",
    "数据切分方式 - 三七分，其中测试集30%，训练集70%，随机种子设置为2018\n",
    "### 任务1：  \n",
    "对数据进行探索和分析。  \n",
    "\n",
    "数据类型的分析  \n",
    "无关特征删除  \n",
    "数据类型转换  \n",
    "缺失值处理  \n",
    "……以及你能想到和借鉴的数据分析处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 一、 观察数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>custid</th>\n",
       "      <th>trade_no</th>\n",
       "      <th>bank_card_no</th>\n",
       "      <th>low_volume_percent</th>\n",
       "      <th>middle_volume_percent</th>\n",
       "      <th>take_amount_in_later_12_month_highest</th>\n",
       "      <th>trans_amount_increase_rate_lately</th>\n",
       "      <th>trans_activity_month</th>\n",
       "      <th>trans_activity_day</th>\n",
       "      <th>...</th>\n",
       "      <th>loans_max_limit</th>\n",
       "      <th>loans_avg_limit</th>\n",
       "      <th>consfin_credit_limit</th>\n",
       "      <th>consfin_credibility</th>\n",
       "      <th>consfin_org_count_current</th>\n",
       "      <th>consfin_product_count</th>\n",
       "      <th>consfin_max_limit</th>\n",
       "      <th>consfin_avg_limit</th>\n",
       "      <th>latest_query_day</th>\n",
       "      <th>loans_latest_day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "      <td>2791858</td>\n",
       "      <td>20180507115231274000000023057383</td>\n",
       "      <td>卡号1</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0</td>\n",
       "      <td>0.90</td>\n",
       "      <td>0.55</td>\n",
       "      <td>0.313</td>\n",
       "      <td>...</td>\n",
       "      <td>2900.0</td>\n",
       "      <td>1688.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>18.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10</td>\n",
       "      <td>534047</td>\n",
       "      <td>20180507121002192000000023073000</td>\n",
       "      <td>卡号1</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.94</td>\n",
       "      <td>2000</td>\n",
       "      <td>1.28</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.458</td>\n",
       "      <td>...</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>1758.0</td>\n",
       "      <td>15100.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>22800.0</td>\n",
       "      <td>9360.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>12</td>\n",
       "      <td>2849787</td>\n",
       "      <td>20180507125159718000000023114911</td>\n",
       "      <td>卡号1</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.114</td>\n",
       "      <td>...</td>\n",
       "      <td>1600.0</td>\n",
       "      <td>1250.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>13</td>\n",
       "      <td>1809708</td>\n",
       "      <td>20180507121358683000000388283484</td>\n",
       "      <td>卡号1</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.96</td>\n",
       "      <td>2000</td>\n",
       "      <td>0.13</td>\n",
       "      <td>0.57</td>\n",
       "      <td>0.777</td>\n",
       "      <td>...</td>\n",
       "      <td>3200.0</td>\n",
       "      <td>1541.0</td>\n",
       "      <td>16300.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>30000.0</td>\n",
       "      <td>12180.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>2499829</td>\n",
       "      <td>20180507115448545000000388205844</td>\n",
       "      <td>卡号1</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0</td>\n",
       "      <td>0.46</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.175</td>\n",
       "      <td>...</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>1630.0</td>\n",
       "      <td>8300.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>8400.0</td>\n",
       "      <td>8250.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>120.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 90 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0   custid                          trade_no bank_card_no  \\\n",
       "0           5  2791858  20180507115231274000000023057383          卡号1   \n",
       "1          10   534047  20180507121002192000000023073000          卡号1   \n",
       "2          12  2849787  20180507125159718000000023114911          卡号1   \n",
       "3          13  1809708  20180507121358683000000388283484          卡号1   \n",
       "4          14  2499829  20180507115448545000000388205844          卡号1   \n",
       "\n",
       "   low_volume_percent  middle_volume_percent  \\\n",
       "0                0.01                   0.99   \n",
       "1                0.02                   0.94   \n",
       "2                0.04                   0.96   \n",
       "3                0.00                   0.96   \n",
       "4                0.01                   0.99   \n",
       "\n",
       "   take_amount_in_later_12_month_highest  trans_amount_increase_rate_lately  \\\n",
       "0                                      0                               0.90   \n",
       "1                                   2000                               1.28   \n",
       "2                                      0                               1.00   \n",
       "3                                   2000                               0.13   \n",
       "4                                      0                               0.46   \n",
       "\n",
       "   trans_activity_month  trans_activity_day        ...         \\\n",
       "0                  0.55               0.313        ...          \n",
       "1                  1.00               0.458        ...          \n",
       "2                  1.00               0.114        ...          \n",
       "3                  0.57               0.777        ...          \n",
       "4                  1.00               0.175        ...          \n",
       "\n",
       "   loans_max_limit  loans_avg_limit  consfin_credit_limit  \\\n",
       "0           2900.0           1688.0                1200.0   \n",
       "1           3500.0           1758.0               15100.0   \n",
       "2           1600.0           1250.0                4200.0   \n",
       "3           3200.0           1541.0               16300.0   \n",
       "4           2300.0           1630.0                8300.0   \n",
       "\n",
       "   consfin_credibility  consfin_org_count_current  consfin_product_count  \\\n",
       "0                 75.0                        1.0                    2.0   \n",
       "1                 80.0                        5.0                    6.0   \n",
       "2                 87.0                        1.0                    1.0   \n",
       "3                 80.0                        5.0                    5.0   \n",
       "4                 79.0                        2.0                    2.0   \n",
       "\n",
       "   consfin_max_limit  consfin_avg_limit  latest_query_day  loans_latest_day  \n",
       "0             1200.0             1200.0              12.0              18.0  \n",
       "1            22800.0             9360.0               4.0               2.0  \n",
       "2             4200.0             4200.0               2.0               6.0  \n",
       "3            30000.0            12180.0               2.0               4.0  \n",
       "4             8400.0             8250.0              22.0             120.0  \n",
       "\n",
       "[5 rows x 90 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "\n",
    "df = pd.read_csv('data.csv',encoding = 'gbk')\n",
    "df.head(5) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4754 entries, 0 to 4753\n",
      "Data columns (total 90 columns):\n",
      "Unnamed: 0                                    4754 non-null int64\n",
      "custid                                        4754 non-null int64\n",
      "trade_no                                      4754 non-null object\n",
      "bank_card_no                                  4754 non-null object\n",
      "low_volume_percent                            4752 non-null float64\n",
      "middle_volume_percent                         4752 non-null float64\n",
      "take_amount_in_later_12_month_highest         4754 non-null int64\n",
      "trans_amount_increase_rate_lately             4751 non-null float64\n",
      "trans_activity_month                          4752 non-null float64\n",
      "trans_activity_day                            4752 non-null float64\n",
      "transd_mcc                                    4752 non-null float64\n",
      "trans_days_interval_filter                    4746 non-null float64\n",
      "trans_days_interval                           4752 non-null float64\n",
      "regional_mobility                             4752 non-null float64\n",
      "student_feature                               1756 non-null float64\n",
      "repayment_capability                          4754 non-null int64\n",
      "is_high_user                                  4754 non-null int64\n",
      "number_of_trans_from_2011                     4752 non-null float64\n",
      "first_transaction_time                        4752 non-null float64\n",
      "historical_trans_amount                       4754 non-null int64\n",
      "historical_trans_day                          4752 non-null float64\n",
      "rank_trad_1_month                             4752 non-null float64\n",
      "trans_amount_3_month                          4754 non-null int64\n",
      "avg_consume_less_12_valid_month               4752 non-null float64\n",
      "abs                                           4754 non-null int64\n",
      "top_trans_count_last_1_month                  4752 non-null float64\n",
      "avg_price_last_12_month                       4754 non-null int64\n",
      "avg_price_top_last_12_valid_month             4650 non-null float64\n",
      "reg_preference_for_trad                       4752 non-null object\n",
      "trans_top_time_last_1_month                   4746 non-null float64\n",
      "trans_top_time_last_6_month                   4746 non-null float64\n",
      "consume_top_time_last_1_month                 4746 non-null float64\n",
      "consume_top_time_last_6_month                 4746 non-null float64\n",
      "cross_consume_count_last_1_month              4328 non-null float64\n",
      "trans_fail_top_count_enum_last_1_month        4738 non-null float64\n",
      "trans_fail_top_count_enum_last_6_month        4738 non-null float64\n",
      "trans_fail_top_count_enum_last_12_month       4738 non-null float64\n",
      "consume_mini_time_last_1_month                4728 non-null float64\n",
      "max_cumulative_consume_later_1_month          4754 non-null int64\n",
      "max_consume_count_later_6_month               4746 non-null float64\n",
      "railway_consume_count_last_12_month           4742 non-null float64\n",
      "pawns_auctions_trusts_consume_last_1_month    4754 non-null int64\n",
      "pawns_auctions_trusts_consume_last_6_month    4754 non-null int64\n",
      "jewelry_consume_count_last_6_month            4742 non-null float64\n",
      "status                                        4754 non-null int64\n",
      "source                                        4754 non-null object\n",
      "first_transaction_day                         4752 non-null float64\n",
      "trans_day_last_12_month                       4752 non-null float64\n",
      "id_name                                       4478 non-null object\n",
      "apply_score                                   4450 non-null float64\n",
      "apply_credibility                             4450 non-null float64\n",
      "query_org_count                               4450 non-null float64\n",
      "query_finance_count                           4450 non-null float64\n",
      "query_cash_count                              4450 non-null float64\n",
      "query_sum_count                               4450 non-null float64\n",
      "latest_query_time                             4450 non-null object\n",
      "latest_one_month_apply                        4450 non-null float64\n",
      "latest_three_month_apply                      4450 non-null float64\n",
      "latest_six_month_apply                        4450 non-null float64\n",
      "loans_score                                   4457 non-null float64\n",
      "loans_credibility_behavior                    4457 non-null float64\n",
      "loans_count                                   4457 non-null float64\n",
      "loans_settle_count                            4457 non-null float64\n",
      "loans_overdue_count                           4457 non-null float64\n",
      "loans_org_count_behavior                      4457 non-null float64\n",
      "consfin_org_count_behavior                    4457 non-null float64\n",
      "loans_cash_count                              4457 non-null float64\n",
      "latest_one_month_loan                         4457 non-null float64\n",
      "latest_three_month_loan                       4457 non-null float64\n",
      "latest_six_month_loan                         4457 non-null float64\n",
      "history_suc_fee                               4457 non-null float64\n",
      "history_fail_fee                              4457 non-null float64\n",
      "latest_one_month_suc                          4457 non-null float64\n",
      "latest_one_month_fail                         4457 non-null float64\n",
      "loans_long_time                               4457 non-null float64\n",
      "loans_latest_time                             4457 non-null object\n",
      "loans_credit_limit                            4457 non-null float64\n",
      "loans_credibility_limit                       4457 non-null float64\n",
      "loans_org_count_current                       4457 non-null float64\n",
      "loans_product_count                           4457 non-null float64\n",
      "loans_max_limit                               4457 non-null float64\n",
      "loans_avg_limit                               4457 non-null float64\n",
      "consfin_credit_limit                          4457 non-null float64\n",
      "consfin_credibility                           4457 non-null float64\n",
      "consfin_org_count_current                     4457 non-null float64\n",
      "consfin_product_count                         4457 non-null float64\n",
      "consfin_max_limit                             4457 non-null float64\n",
      "consfin_avg_limit                             4457 non-null float64\n",
      "latest_query_day                              4450 non-null float64\n",
      "loans_latest_day                              4457 non-null float64\n",
      "dtypes: float64(70), int64(13), object(7)\n",
      "memory usage: 3.3+ MB\n"
     ]
    }
   ],
   "source": [
    "df.info() #返回df的所有信息\n",
    "#我们可以知道 有4754条数据，有90个特征（列名）和他们的类别（有些特征的数据不够4754，但是也没看到空值这是咋回事？）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>custid</th>\n",
       "      <th>low_volume_percent</th>\n",
       "      <th>middle_volume_percent</th>\n",
       "      <th>take_amount_in_later_12_month_highest</th>\n",
       "      <th>trans_amount_increase_rate_lately</th>\n",
       "      <th>trans_activity_month</th>\n",
       "      <th>trans_activity_day</th>\n",
       "      <th>transd_mcc</th>\n",
       "      <th>trans_days_interval_filter</th>\n",
       "      <th>...</th>\n",
       "      <th>loans_max_limit</th>\n",
       "      <th>loans_avg_limit</th>\n",
       "      <th>consfin_credit_limit</th>\n",
       "      <th>consfin_credibility</th>\n",
       "      <th>consfin_org_count_current</th>\n",
       "      <th>consfin_product_count</th>\n",
       "      <th>consfin_max_limit</th>\n",
       "      <th>consfin_avg_limit</th>\n",
       "      <th>latest_query_day</th>\n",
       "      <th>loans_latest_day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4754.000000</td>\n",
       "      <td>4.754000e+03</td>\n",
       "      <td>4752.000000</td>\n",
       "      <td>4752.000000</td>\n",
       "      <td>4754.000000</td>\n",
       "      <td>4751.000000</td>\n",
       "      <td>4752.000000</td>\n",
       "      <td>4752.000000</td>\n",
       "      <td>4752.000000</td>\n",
       "      <td>4746.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "      <td>4450.000000</td>\n",
       "      <td>4457.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6008.414178</td>\n",
       "      <td>1.690993e+06</td>\n",
       "      <td>0.021806</td>\n",
       "      <td>0.901294</td>\n",
       "      <td>1940.197728</td>\n",
       "      <td>14.160674</td>\n",
       "      <td>0.804411</td>\n",
       "      <td>0.365425</td>\n",
       "      <td>17.502946</td>\n",
       "      <td>29.029920</td>\n",
       "      <td>...</td>\n",
       "      <td>3390.038142</td>\n",
       "      <td>1820.357864</td>\n",
       "      <td>9187.009199</td>\n",
       "      <td>76.042630</td>\n",
       "      <td>4.732331</td>\n",
       "      <td>5.227507</td>\n",
       "      <td>16153.690823</td>\n",
       "      <td>8007.696881</td>\n",
       "      <td>24.112809</td>\n",
       "      <td>55.181512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3452.071428</td>\n",
       "      <td>1.034235e+06</td>\n",
       "      <td>0.041527</td>\n",
       "      <td>0.144856</td>\n",
       "      <td>3923.971494</td>\n",
       "      <td>694.180473</td>\n",
       "      <td>0.196920</td>\n",
       "      <td>0.170196</td>\n",
       "      <td>4.475616</td>\n",
       "      <td>22.722432</td>\n",
       "      <td>...</td>\n",
       "      <td>1474.206546</td>\n",
       "      <td>583.418291</td>\n",
       "      <td>7371.257043</td>\n",
       "      <td>14.536819</td>\n",
       "      <td>2.974596</td>\n",
       "      <td>3.409292</td>\n",
       "      <td>14301.037628</td>\n",
       "      <td>5679.418585</td>\n",
       "      <td>37.725724</td>\n",
       "      <td>53.486408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.140000e+02</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>0.033000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-2.000000</td>\n",
       "      <td>-2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3106.000000</td>\n",
       "      <td>7.593358e+05</td>\n",
       "      <td>0.010000</td>\n",
       "      <td>0.880000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.615000</td>\n",
       "      <td>0.670000</td>\n",
       "      <td>0.233000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>2300.000000</td>\n",
       "      <td>1535.000000</td>\n",
       "      <td>4800.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>7800.000000</td>\n",
       "      <td>4737.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>10.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6006.500000</td>\n",
       "      <td>1.634942e+06</td>\n",
       "      <td>0.010000</td>\n",
       "      <td>0.960000</td>\n",
       "      <td>500.000000</td>\n",
       "      <td>0.970000</td>\n",
       "      <td>0.860000</td>\n",
       "      <td>0.350000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>3100.000000</td>\n",
       "      <td>1810.000000</td>\n",
       "      <td>7700.000000</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>13800.000000</td>\n",
       "      <td>7050.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>36.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>8999.000000</td>\n",
       "      <td>2.597905e+06</td>\n",
       "      <td>0.020000</td>\n",
       "      <td>0.990000</td>\n",
       "      <td>2000.000000</td>\n",
       "      <td>1.600000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.480000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>4300.000000</td>\n",
       "      <td>2100.000000</td>\n",
       "      <td>11700.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>20400.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>91.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>11992.000000</td>\n",
       "      <td>4.004694e+06</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>68000.000000</td>\n",
       "      <td>47596.740000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.941000</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>285.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>6900.000000</td>\n",
       "      <td>87100.000000</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>266400.000000</td>\n",
       "      <td>82800.000000</td>\n",
       "      <td>360.000000</td>\n",
       "      <td>323.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 83 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0        custid  low_volume_percent  middle_volume_percent  \\\n",
       "count   4754.000000  4.754000e+03         4752.000000            4752.000000   \n",
       "mean    6008.414178  1.690993e+06            0.021806               0.901294   \n",
       "std     3452.071428  1.034235e+06            0.041527               0.144856   \n",
       "min        5.000000  1.140000e+02            0.000000               0.000000   \n",
       "25%     3106.000000  7.593358e+05            0.010000               0.880000   \n",
       "50%     6006.500000  1.634942e+06            0.010000               0.960000   \n",
       "75%     8999.000000  2.597905e+06            0.020000               0.990000   \n",
       "max    11992.000000  4.004694e+06            1.000000               1.000000   \n",
       "\n",
       "       take_amount_in_later_12_month_highest  \\\n",
       "count                            4754.000000   \n",
       "mean                             1940.197728   \n",
       "std                              3923.971494   \n",
       "min                                 0.000000   \n",
       "25%                                 0.000000   \n",
       "50%                               500.000000   \n",
       "75%                              2000.000000   \n",
       "max                             68000.000000   \n",
       "\n",
       "       trans_amount_increase_rate_lately  trans_activity_month  \\\n",
       "count                        4751.000000           4752.000000   \n",
       "mean                           14.160674              0.804411   \n",
       "std                           694.180473              0.196920   \n",
       "min                             0.000000              0.120000   \n",
       "25%                             0.615000              0.670000   \n",
       "50%                             0.970000              0.860000   \n",
       "75%                             1.600000              1.000000   \n",
       "max                         47596.740000              1.000000   \n",
       "\n",
       "       trans_activity_day   transd_mcc  trans_days_interval_filter  \\\n",
       "count         4752.000000  4752.000000                 4746.000000   \n",
       "mean             0.365425    17.502946                   29.029920   \n",
       "std              0.170196     4.475616                   22.722432   \n",
       "min              0.033000     2.000000                    0.000000   \n",
       "25%              0.233000    15.000000                   16.000000   \n",
       "50%              0.350000    17.000000                   23.000000   \n",
       "75%              0.480000    20.000000                   32.000000   \n",
       "max              0.941000    42.000000                  285.000000   \n",
       "\n",
       "             ...         loans_max_limit  loans_avg_limit  \\\n",
       "count        ...             4457.000000      4457.000000   \n",
       "mean         ...             3390.038142      1820.357864   \n",
       "std          ...             1474.206546       583.418291   \n",
       "min          ...                0.000000         0.000000   \n",
       "25%          ...             2300.000000      1535.000000   \n",
       "50%          ...             3100.000000      1810.000000   \n",
       "75%          ...             4300.000000      2100.000000   \n",
       "max          ...            10000.000000      6900.000000   \n",
       "\n",
       "       consfin_credit_limit  consfin_credibility  consfin_org_count_current  \\\n",
       "count           4457.000000          4457.000000                4457.000000   \n",
       "mean            9187.009199            76.042630                   4.732331   \n",
       "std             7371.257043            14.536819                   2.974596   \n",
       "min                0.000000             0.000000                   0.000000   \n",
       "25%             4800.000000            77.000000                   2.000000   \n",
       "50%             7700.000000            79.000000                   4.000000   \n",
       "75%            11700.000000            80.000000                   7.000000   \n",
       "max            87100.000000            87.000000                  18.000000   \n",
       "\n",
       "       consfin_product_count  consfin_max_limit  consfin_avg_limit  \\\n",
       "count            4457.000000        4457.000000        4457.000000   \n",
       "mean                5.227507       16153.690823        8007.696881   \n",
       "std                 3.409292       14301.037628        5679.418585   \n",
       "min                 0.000000           0.000000           0.000000   \n",
       "25%                 3.000000        7800.000000        4737.000000   \n",
       "50%                 5.000000       13800.000000        7050.000000   \n",
       "75%                 7.000000       20400.000000       10000.000000   \n",
       "max                20.000000      266400.000000       82800.000000   \n",
       "\n",
       "       latest_query_day  loans_latest_day  \n",
       "count       4450.000000       4457.000000  \n",
       "mean          24.112809         55.181512  \n",
       "std           37.725724         53.486408  \n",
       "min           -2.000000         -2.000000  \n",
       "25%            5.000000         10.000000  \n",
       "50%           14.000000         36.000000  \n",
       "75%           24.000000         91.000000  \n",
       "max          360.000000        323.000000  \n",
       "\n",
       "[8 rows x 83 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe() #可以看到统计信息"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "we have 70 columns in type float64, they are ['low_volume_percent', 'middle_volume_percent', 'trans_amount_increase_rate_lately', 'trans_activity_month', 'trans_activity_day', 'transd_mcc', 'trans_days_interval_filter', 'trans_days_interval', 'regional_mobility', 'student_feature', 'number_of_trans_from_2011', 'first_transaction_time', 'historical_trans_day', 'rank_trad_1_month', 'avg_consume_less_12_valid_month', 'top_trans_count_last_1_month', 'avg_price_top_last_12_valid_month', 'trans_top_time_last_1_month', 'trans_top_time_last_6_month', 'consume_top_time_last_1_month', 'consume_top_time_last_6_month', 'cross_consume_count_last_1_month', 'trans_fail_top_count_enum_last_1_month', 'trans_fail_top_count_enum_last_6_month', 'trans_fail_top_count_enum_last_12_month', 'consume_mini_time_last_1_month', 'max_consume_count_later_6_month', 'railway_consume_count_last_12_month', 'jewelry_consume_count_last_6_month', 'first_transaction_day', 'trans_day_last_12_month', 'apply_score', 'apply_credibility', 'query_org_count', 'query_finance_count', 'query_cash_count', 'query_sum_count', 'latest_one_month_apply', 'latest_three_month_apply', 'latest_six_month_apply', 'loans_score', 'loans_credibility_behavior', 'loans_count', 'loans_settle_count', 'loans_overdue_count', 'loans_org_count_behavior', 'consfin_org_count_behavior', 'loans_cash_count', 'latest_one_month_loan', 'latest_three_month_loan', 'latest_six_month_loan', 'history_suc_fee', 'history_fail_fee', 'latest_one_month_suc', 'latest_one_month_fail', 'loans_long_time', 'loans_credit_limit', 'loans_credibility_limit', 'loans_org_count_current', 'loans_product_count', 'loans_max_limit', 'loans_avg_limit', 'consfin_credit_limit', 'consfin_credibility', 'consfin_org_count_current', 'consfin_product_count', 'consfin_max_limit', 'consfin_avg_limit', 'latest_query_day', 'loans_latest_day']\\\n",
      "we have 13 columns in type int64, they are ['Unnamed: 0', 'custid', 'take_amount_in_later_12_month_highest', 'repayment_capability', 'is_high_user', 'historical_trans_amount', 'trans_amount_3_month', 'abs', 'avg_price_last_12_month', 'max_cumulative_consume_later_1_month', 'pawns_auctions_trusts_consume_last_1_month', 'pawns_auctions_trusts_consume_last_6_month', 'status']\\\n",
      "we have 7 columns in type object, they are ['trade_no', 'bank_card_no', 'reg_preference_for_trad', 'source', 'id_name', 'latest_query_time', 'loans_latest_time']\\\n"
     ]
    }
   ],
   "source": [
    "#提取我们需要的信息，特别是每个特征的类别信息\n",
    "\n",
    "def get_data_type(df):\n",
    "    typedic= {} # 类型字典\n",
    "    for name in df.columns:\n",
    "        typedic[str(df[name].dtype)] = typedic.get(str(df[name].dtype),[])+[name]\n",
    "    for key,value in typedic.items():\n",
    "        # print('we have {} columns in type {}'.format(len(value),key))\n",
    "        print('we have {} columns in type {}, they are {}\\\\'.format(len(value),key,value))\n",
    "\n",
    "get_data_type(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 二、数据预处理  \n",
    "无关特征删除  \n",
    "数据类型转换  \n",
    "缺失值处理等等"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1 .  无关特征删除（每个特征过一下）  \n",
    "有90个特征,选出应该去除的\n",
    "\n",
    "先对 类型为obect的查看一下：['trade_no', 'bank_card_no', 'reg_preference_for_trad', 'source', 'id_name', 'latest_query_time', 'loans_latest_time']\n",
    "'bank_card_no' ：都是卡号一，去掉\n",
    "'id_name'：客户名字，去掉\n",
    "\n",
    "\n",
    "Unnamed: 0： 应该是原来的数据序号，删掉一些无用数据，造成序号不连续。可以去掉  \n",
    "custid : 顾客id号没啥分析\n",
    " \n",
    " \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Unnamed: 0', 'custid', 'trade_no', 'bank_card_no',\n",
       "       'low_volume_percent', 'middle_volume_percent',\n",
       "       'take_amount_in_later_12_month_highest',\n",
       "       'trans_amount_increase_rate_lately', 'trans_activity_month',\n",
       "       'trans_activity_day', 'transd_mcc', 'trans_days_interval_filter',\n",
       "       'trans_days_interval', 'regional_mobility', 'student_feature',\n",
       "       'repayment_capability', 'is_high_user', 'number_of_trans_from_2011',\n",
       "       'first_transaction_time', 'historical_trans_amount',\n",
       "       'historical_trans_day', 'rank_trad_1_month', 'trans_amount_3_month',\n",
       "       'avg_consume_less_12_valid_month', 'abs',\n",
       "       'top_trans_count_last_1_month', 'avg_price_last_12_month',\n",
       "       'avg_price_top_last_12_valid_month', 'reg_preference_for_trad',\n",
       "       'trans_top_time_last_1_month', 'trans_top_time_last_6_month',\n",
       "       'consume_top_time_last_1_month', 'consume_top_time_last_6_month',\n",
       "       'cross_consume_count_last_1_month',\n",
       "       'trans_fail_top_count_enum_last_1_month',\n",
       "       'trans_fail_top_count_enum_last_6_month',\n",
       "       'trans_fail_top_count_enum_last_12_month',\n",
       "       'consume_mini_time_last_1_month',\n",
       "       'max_cumulative_consume_later_1_month',\n",
       "       'max_consume_count_later_6_month',\n",
       "       'railway_consume_count_last_12_month',\n",
       "       'pawns_auctions_trusts_consume_last_1_month',\n",
       "       'pawns_auctions_trusts_consume_last_6_month',\n",
       "       'jewelry_consume_count_last_6_month', 'status', 'source',\n",
       "       'first_transaction_day', 'trans_day_last_12_month', 'id_name',\n",
       "       'apply_score', 'apply_credibility', 'query_org_count',\n",
       "       'query_finance_count', 'query_cash_count', 'query_sum_count',\n",
       "       'latest_query_time', 'latest_one_month_apply',\n",
       "       'latest_three_month_apply', 'latest_six_month_apply', 'loans_score',\n",
       "       'loans_credibility_behavior', 'loans_count', 'loans_settle_count',\n",
       "       'loans_overdue_count', 'loans_org_count_behavior',\n",
       "       'consfin_org_count_behavior', 'loans_cash_count',\n",
       "       'latest_one_month_loan', 'latest_three_month_loan',\n",
       "       'latest_six_month_loan', 'history_suc_fee', 'history_fail_fee',\n",
       "       'latest_one_month_suc', 'latest_one_month_fail', 'loans_long_time',\n",
       "       'loans_latest_time', 'loans_credit_limit', 'loans_credibility_limit',\n",
       "       'loans_org_count_current', 'loans_product_count', 'loans_max_limit',\n",
       "       'loans_avg_limit', 'consfin_credit_limit', 'consfin_credibility',\n",
       "       'consfin_org_count_current', 'consfin_product_count',\n",
       "       'consfin_max_limit', 'consfin_avg_limit', 'latest_query_day',\n",
       "       'loans_latest_day'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    20180507115231274000000023057383\n",
       "1    20180507121002192000000023073000\n",
       "2    20180507125159718000000023114911\n",
       "3    20180507121358683000000388283484\n",
       "4    20180507115448545000000388205844\n",
       "Name: trade_no, dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['trade_no'].head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "交易号，有时间信息，但后面也还有时间的特征，这里可以去掉"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    卡号1\n",
       "1    卡号1\n",
       "2    卡号1\n",
       "3    卡号1\n",
       "4    卡号1\n",
       "Name: bank_card_no, dtype: object"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['bank_card_no'].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "xs    4754\n",
       "Name: source, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看重复的值就可以去掉了\n",
    "\n",
    "df['source'].value_counts() #railway_consume_count_last_12_month\n",
    "\n",
    "# df['source'].duplicated().sum() #查看不同元素的数量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "都是一样，删除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0     4651\n",
       "1.0       72\n",
       "2.0       13\n",
       "4.0        3\n",
       "3.0        2\n",
       "30.0       1\n",
       "Name: railway_consume_count_last_12_month, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['railway_consume_count_last_12_month'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 综上，删掉的值有'Unnamed: 0','custid','trade_no','bank_card_no' ,'source','id_name'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "84\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Index(['low_volume_percent', 'middle_volume_percent',\n",
       "       'take_amount_in_later_12_month_highest',\n",
       "       'trans_amount_increase_rate_lately', 'trans_activity_month',\n",
       "       'trans_activity_day', 'transd_mcc', 'trans_days_interval_filter',\n",
       "       'trans_days_interval', 'regional_mobility', 'student_feature',\n",
       "       'repayment_capability', 'is_high_user', 'number_of_trans_from_2011',\n",
       "       'first_transaction_time', 'historical_trans_amount',\n",
       "       'historical_trans_day', 'rank_trad_1_month', 'trans_amount_3_month',\n",
       "       'avg_consume_less_12_valid_month', 'abs',\n",
       "       'top_trans_count_last_1_month', 'avg_price_last_12_month',\n",
       "       'avg_price_top_last_12_valid_month', 'reg_preference_for_trad',\n",
       "       'trans_top_time_last_1_month', 'trans_top_time_last_6_month',\n",
       "       'consume_top_time_last_1_month', 'consume_top_time_last_6_month',\n",
       "       'cross_consume_count_last_1_month',\n",
       "       'trans_fail_top_count_enum_last_1_month',\n",
       "       'trans_fail_top_count_enum_last_6_month',\n",
       "       'trans_fail_top_count_enum_last_12_month',\n",
       "       'consume_mini_time_last_1_month',\n",
       "       'max_cumulative_consume_later_1_month',\n",
       "       'max_consume_count_later_6_month',\n",
       "       'railway_consume_count_last_12_month',\n",
       "       'pawns_auctions_trusts_consume_last_1_month',\n",
       "       'pawns_auctions_trusts_consume_last_6_month',\n",
       "       'jewelry_consume_count_last_6_month', 'status', 'first_transaction_day',\n",
       "       'trans_day_last_12_month', 'apply_score', 'apply_credibility',\n",
       "       'query_org_count', 'query_finance_count', 'query_cash_count',\n",
       "       'query_sum_count', 'latest_query_time', 'latest_one_month_apply',\n",
       "       'latest_three_month_apply', 'latest_six_month_apply', 'loans_score',\n",
       "       'loans_credibility_behavior', 'loans_count', 'loans_settle_count',\n",
       "       'loans_overdue_count', 'loans_org_count_behavior',\n",
       "       'consfin_org_count_behavior', 'loans_cash_count',\n",
       "       'latest_one_month_loan', 'latest_three_month_loan',\n",
       "       'latest_six_month_loan', 'history_suc_fee', 'history_fail_fee',\n",
       "       'latest_one_month_suc', 'latest_one_month_fail', 'loans_long_time',\n",
       "       'loans_latest_time', 'loans_credit_limit', 'loans_credibility_limit',\n",
       "       'loans_org_count_current', 'loans_product_count', 'loans_max_limit',\n",
       "       'loans_avg_limit', 'consfin_credit_limit', 'consfin_credibility',\n",
       "       'consfin_org_count_current', 'consfin_product_count',\n",
       "       'consfin_max_limit', 'consfin_avg_limit', 'latest_query_day',\n",
       "       'loans_latest_day'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop(['Unnamed: 0','custid','trade_no','bank_card_no' ,'source','id_name'],inplace =True,axis = 1)\n",
    "print(len(df.columns))\n",
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 2.数值类型转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "we have 3 columns in type object, they are ['reg_preference_for_trad', 'latest_query_time', 'loans_latest_time']\\\n",
      "we have 11 columns in type int64, they are ['take_amount_in_later_12_month_highest', 'repayment_capability', 'is_high_user', 'historical_trans_amount', 'trans_amount_3_month', 'abs', 'avg_price_last_12_month', 'max_cumulative_consume_later_1_month', 'pawns_auctions_trusts_consume_last_1_month', 'pawns_auctions_trusts_consume_last_6_month', 'status']\\\n",
      "we have 70 columns in type float64, they are ['low_volume_percent', 'middle_volume_percent', 'trans_amount_increase_rate_lately', 'trans_activity_month', 'trans_activity_day', 'transd_mcc', 'trans_days_interval_filter', 'trans_days_interval', 'regional_mobility', 'student_feature', 'number_of_trans_from_2011', 'first_transaction_time', 'historical_trans_day', 'rank_trad_1_month', 'avg_consume_less_12_valid_month', 'top_trans_count_last_1_month', 'avg_price_top_last_12_valid_month', 'trans_top_time_last_1_month', 'trans_top_time_last_6_month', 'consume_top_time_last_1_month', 'consume_top_time_last_6_month', 'cross_consume_count_last_1_month', 'trans_fail_top_count_enum_last_1_month', 'trans_fail_top_count_enum_last_6_month', 'trans_fail_top_count_enum_last_12_month', 'consume_mini_time_last_1_month', 'max_consume_count_later_6_month', 'railway_consume_count_last_12_month', 'jewelry_consume_count_last_6_month', 'first_transaction_day', 'trans_day_last_12_month', 'apply_score', 'apply_credibility', 'query_org_count', 'query_finance_count', 'query_cash_count', 'query_sum_count', 'latest_one_month_apply', 'latest_three_month_apply', 'latest_six_month_apply', 'loans_score', 'loans_credibility_behavior', 'loans_count', 'loans_settle_count', 'loans_overdue_count', 'loans_org_count_behavior', 'consfin_org_count_behavior', 'loans_cash_count', 'latest_one_month_loan', 'latest_three_month_loan', 'latest_six_month_loan', 'history_suc_fee', 'history_fail_fee', 'latest_one_month_suc', 'latest_one_month_fail', 'loans_long_time', 'loans_credit_limit', 'loans_credibility_limit', 'loans_org_count_current', 'loans_product_count', 'loans_max_limit', 'loans_avg_limit', 'consfin_credit_limit', 'consfin_credibility', 'consfin_org_count_current', 'consfin_product_count', 'consfin_max_limit', 'consfin_avg_limit', 'latest_query_day', 'loans_latest_day']\\\n"
     ]
    }
   ],
   "source": [
    "get_data_type(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 非数值型的有三个 ['reg_preference_for_trad', 'latest_query_time', 'loans_latest_time']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "一线城市    3403\n",
       "三线城市    1064\n",
       "境外       150\n",
       "二线城市     131\n",
       "其他城市       4\n",
       "NaN        2\n",
       "Name: reg_preference_for_trad, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 城市类型\n",
    "df['reg_preference_for_trad'].value_counts(dropna = False) #"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将未知的填充为其他城市，并且转为数字"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "df['reg_preference_for_trad'].fillna('其他城市',inplace = True)\n",
    "df['reg_preference_for_trad'].replace({'一线城市':1,'二线城市':2,'三线城市':3,'境外':4,'其他城市':5},inplace = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "'latest_query_time', 'loans_latest_time'为日期类型，先不动"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.缺失值处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "low_volume_percent                          2\n",
       "middle_volume_percent                       2\n",
       "take_amount_in_later_12_month_highest       0\n",
       "trans_amount_increase_rate_lately           3\n",
       "trans_activity_month                        2\n",
       "trans_activity_day                          2\n",
       "transd_mcc                                  2\n",
       "trans_days_interval_filter                  8\n",
       "trans_days_interval                         2\n",
       "regional_mobility                           2\n",
       "student_feature                          2998\n",
       "repayment_capability                        0\n",
       "is_high_user                                0\n",
       "number_of_trans_from_2011                   2\n",
       "first_transaction_time                      2\n",
       "historical_trans_amount                     0\n",
       "historical_trans_day                        2\n",
       "rank_trad_1_month                           2\n",
       "trans_amount_3_month                        0\n",
       "avg_consume_less_12_valid_month             2\n",
       "abs                                         0\n",
       "top_trans_count_last_1_month                2\n",
       "avg_price_last_12_month                     0\n",
       "avg_price_top_last_12_valid_month         104\n",
       "reg_preference_for_trad                     0\n",
       "trans_top_time_last_1_month                 8\n",
       "trans_top_time_last_6_month                 8\n",
       "consume_top_time_last_1_month               8\n",
       "consume_top_time_last_6_month               8\n",
       "cross_consume_count_last_1_month          426\n",
       "                                         ... \n",
       "loans_credibility_behavior                297\n",
       "loans_count                               297\n",
       "loans_settle_count                        297\n",
       "loans_overdue_count                       297\n",
       "loans_org_count_behavior                  297\n",
       "consfin_org_count_behavior                297\n",
       "loans_cash_count                          297\n",
       "latest_one_month_loan                     297\n",
       "latest_three_month_loan                   297\n",
       "latest_six_month_loan                     297\n",
       "history_suc_fee                           297\n",
       "history_fail_fee                          297\n",
       "latest_one_month_suc                      297\n",
       "latest_one_month_fail                     297\n",
       "loans_long_time                           297\n",
       "loans_latest_time                         297\n",
       "loans_credit_limit                        297\n",
       "loans_credibility_limit                   297\n",
       "loans_org_count_current                   297\n",
       "loans_product_count                       297\n",
       "loans_max_limit                           297\n",
       "loans_avg_limit                           297\n",
       "consfin_credit_limit                      297\n",
       "consfin_credibility                       297\n",
       "consfin_org_count_current                 297\n",
       "consfin_product_count                     297\n",
       "consfin_max_limit                         297\n",
       "consfin_avg_limit                         297\n",
       "latest_query_day                          304\n",
       "loans_latest_day                          297\n",
       "Length: 84, dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum() #查看缺失值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "学生缺的比较多"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NaN     2998\n",
       " 1.0    1754\n",
       " 2.0       2\n",
       "Name: student_feature, dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['student_feature'].value_counts(dropna=False) #True会默认把缺失值或者na，null这些值去掉"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 这里可能是不是学生的意思，暂时NA看做是0吧"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    2998\n",
       "1.0    1754\n",
       "2.0       2\n",
       "Name: student_feature, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['student_feature'].fillna(0,inplace=True) #True代码改变了源数据\n",
    "df['student_feature'].value_counts(dropna=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 其余特征缺失的话可以用众数填充  \n",
    "也有说法是  \n",
    "数值型取中位数  \n",
    "日期取众数  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>low_volume_percent</th>\n",
       "      <th>middle_volume_percent</th>\n",
       "      <th>take_amount_in_later_12_month_highest</th>\n",
       "      <th>trans_amount_increase_rate_lately</th>\n",
       "      <th>trans_activity_month</th>\n",
       "      <th>trans_activity_day</th>\n",
       "      <th>transd_mcc</th>\n",
       "      <th>trans_days_interval_filter</th>\n",
       "      <th>trans_days_interval</th>\n",
       "      <th>regional_mobility</th>\n",
       "      <th>...</th>\n",
       "      <th>loans_max_limit</th>\n",
       "      <th>loans_avg_limit</th>\n",
       "      <th>consfin_credit_limit</th>\n",
       "      <th>consfin_credibility</th>\n",
       "      <th>consfin_org_count_current</th>\n",
       "      <th>consfin_product_count</th>\n",
       "      <th>consfin_max_limit</th>\n",
       "      <th>consfin_avg_limit</th>\n",
       "      <th>latest_query_day</th>\n",
       "      <th>loans_latest_day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.01</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0</td>\n",
       "      <td>0.90</td>\n",
       "      <td>0.55</td>\n",
       "      <td>0.313</td>\n",
       "      <td>17.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>2900.0</td>\n",
       "      <td>1688.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>18.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02</td>\n",
       "      <td>0.94</td>\n",
       "      <td>2000</td>\n",
       "      <td>1.28</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.458</td>\n",
       "      <td>19.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>1758.0</td>\n",
       "      <td>15100.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>22800.0</td>\n",
       "      <td>9360.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.04</td>\n",
       "      <td>0.96</td>\n",
       "      <td>0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.114</td>\n",
       "      <td>13.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1600.0</td>\n",
       "      <td>1250.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.96</td>\n",
       "      <td>2000</td>\n",
       "      <td>0.13</td>\n",
       "      <td>0.57</td>\n",
       "      <td>0.777</td>\n",
       "      <td>22.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>3200.0</td>\n",
       "      <td>1541.0</td>\n",
       "      <td>16300.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>30000.0</td>\n",
       "      <td>12180.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.01</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0</td>\n",
       "      <td>0.46</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.175</td>\n",
       "      <td>13.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>1630.0</td>\n",
       "      <td>8300.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>8400.0</td>\n",
       "      <td>8250.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>120.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 84 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   low_volume_percent  middle_volume_percent  \\\n",
       "0                0.01                   0.99   \n",
       "1                0.02                   0.94   \n",
       "2                0.04                   0.96   \n",
       "3                0.00                   0.96   \n",
       "4                0.01                   0.99   \n",
       "\n",
       "   take_amount_in_later_12_month_highest  trans_amount_increase_rate_lately  \\\n",
       "0                                      0                               0.90   \n",
       "1                                   2000                               1.28   \n",
       "2                                      0                               1.00   \n",
       "3                                   2000                               0.13   \n",
       "4                                      0                               0.46   \n",
       "\n",
       "   trans_activity_month  trans_activity_day  transd_mcc  \\\n",
       "0                  0.55               0.313        17.0   \n",
       "1                  1.00               0.458        19.0   \n",
       "2                  1.00               0.114        13.0   \n",
       "3                  0.57               0.777        22.0   \n",
       "4                  1.00               0.175        13.0   \n",
       "\n",
       "   trans_days_interval_filter  trans_days_interval  regional_mobility  \\\n",
       "0                        27.0                 26.0                3.0   \n",
       "1                        30.0                 14.0                4.0   \n",
       "2                        68.0                 22.0                1.0   \n",
       "3                        14.0                  6.0                3.0   \n",
       "4                        66.0                 42.0                1.0   \n",
       "\n",
       "         ...         loans_max_limit  loans_avg_limit  consfin_credit_limit  \\\n",
       "0        ...                  2900.0           1688.0                1200.0   \n",
       "1        ...                  3500.0           1758.0               15100.0   \n",
       "2        ...                  1600.0           1250.0                4200.0   \n",
       "3        ...                  3200.0           1541.0               16300.0   \n",
       "4        ...                  2300.0           1630.0                8300.0   \n",
       "\n",
       "   consfin_credibility  consfin_org_count_current  consfin_product_count  \\\n",
       "0                 75.0                        1.0                    2.0   \n",
       "1                 80.0                        5.0                    6.0   \n",
       "2                 87.0                        1.0                    1.0   \n",
       "3                 80.0                        5.0                    5.0   \n",
       "4                 79.0                        2.0                    2.0   \n",
       "\n",
       "   consfin_max_limit  consfin_avg_limit  latest_query_day  loans_latest_day  \n",
       "0             1200.0             1200.0              12.0              18.0  \n",
       "1            22800.0             9360.0               4.0               2.0  \n",
       "2             4200.0             4200.0               2.0               6.0  \n",
       "3            30000.0            12180.0               2.0               4.0  \n",
       "4             8400.0             8250.0              22.0             120.0  \n",
       "\n",
       "[5 rows x 84 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# print(type(df.columns[1]))\n",
    "for i in df.columns:\n",
    "    df[i].fillna(df[i].mode()[0],inplace = True)  #加[0]是因为众数可能有多个，返回不是一个数字\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 三、数据集切分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3327, 83) (1427, 83) (3327,) (1427,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "Y=df['status']\n",
    "X=df.drop('status',axis=1)\n",
    "X_train,X_test,Y_train,Y_test =train_test_split(X,Y,test_size=0.3,random_state = 2018)   \n",
    "print(X_train.shape, X_test.shape, Y_train.shape, Y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:tensorflow-gpu]",
   "language": "python",
   "name": "conda-env-tensorflow-gpu-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
